A control study is one which uses a control group to compare to an experimental group in a test of a causal hypothesis.
For example, let's say that a friend tells you that he has become involved in manifesting. He claims that anyone can create their own luck and make things happen using only will power. He tells you that there is no such thing as coincidence and that each of us creates our own reality.
At first you may be both repelled and attracted to your friend's notions. The idea of being able to make the world conform to your wishes is very attractive. On the other hand, you have lived in the world long enough to know that "if wishes were fishes we'd all live in the sea." Your experience seems to contradict the claims of manifesting. Good luck and bad luck seem to fall indiscriminantly on people, without rhyme or reason. Good things happen to good and bad people alike. And bad things happen to good and bad people alike. But you know that some people say that so-and-so has "good luck," whereas someone else has "bad luck." Maybe it's not just fate or chance that makes some people lucky and others unlucky.
Experience also tells you that coincidences happen all the time. You run to the dictionary and look up 'coincidence' and you find that the second meaning is "A sequence of events that although accidental seems to have been planned or arranged." You wonder: what if coincidences are really sequences of events that although seemingly accidental are actually planned or arranged?
Finally, you consider the claim that a person can control external events by an internal act of will. That claim you know from experience is preposterous. You have tried this one ever since you first heard of psychokinesis. You have tried to move pencils with your mind, win the lottery by willing numbers to be picked, make a pimple disappear by imagining little pimple killers sucking it into your bloodstream. You've even tried getting girls to be attracted to you by trying to psychically plant desires for you in their minds. Nothing worked. You've heard of people getting together to end war or world hunger by willing it simultaneously, but wars continue and world hunger is as much a problem as ever.
So, part of you is saying that manifesting is just one in a long line of delusional pipedreams, sounding full and furious but signifying nothing. Yet, another part of you is saying that maybe there is something to this one, maybe this is not just another one of those too-good-to-be-true scams. One thing in its favor, you think, is that your friend has not mentioned paranormal or divine forces at work in manifesting. He has only spoken of the power of the will. This might be testable, you think.
But how do you test manifesting? First, you know you must clarify your ideas so that you know exactly what it is you are testing. What claim would you test?
"A person can create his own reality."
"A person can make events happen by an act of will."
"Anyone can create their own luck."
"There is no such thing as coincidence."
"All events are the result of will power."
"All events are part of a plan."
To help you decide what claim to test, you must consider what a control test aims to do. You want to be able to compare two groups, a control group with an experimental group. You want the experimental group to use manifesting techniques in order to achieve some specified, measurable or observable effect. You want the control group to be no different in any significant way from the experimental group except in the fact that the controls do not use manifesting. You will reason that if there is a significant difference in outcome between the control and experimental groups, it will most likely be because of the presence or absence of manifesting in the two groups.
The reason you want to do a control study is because if you only study those who use manifesting, you will not be able to rule out other possible causes of any effects you observe. This principle might be easier to understand if we another example. Let's say that you have a few pimples on your face and you want to get rid of them. There are a number of "blemish medicines" on the shelves of your local pharmacy. One, let's call it ZitsRgone, promises to get rid of unsightly blemishes in seven days or your money back. You buy it and try it and sure enough in seven days your pimples are gone. Did the ZitsRgone cause your pimples to go away? You can't really be sure. They might have gone away if you had done nothing special to your face. To be sure that the ZitsRgone is effective treatment, you need to do a control study.
Let's say that you have six pimples on each cheek. The pimples are very similar, equal size pustules, etc. You put ZitsRgone on one cheek (your experimental group of pimples) and nothing on the other (the control group). If ZitsRgone is a significant causal factor in pimple eradication, then you should notice a significant difference in pimple population at the end of your experiment. If, after a week, both sides seem to be pretty much the same, then it is likely that ZitsRgone is not a significant causal factor here. If, however, the ZitsRgone side is all cleared up and the other side is still blemished, may you now proclaim that ZitsRgone works? Unfortunately, no. It is possible that you favored the ZitsRgone side and your perception has been affected by your desire for the stuff to work. To test your hypothesis properly you need to do a double-blind study.
To eliminate the possibility of bias in perception and evaluation, as well as in treatment, we must give you two creams, one which contains ZitsRgone, and another which looks like ZitsRgone and is applied in the same way but is actually a placebo. We will not tell you which cream is the ZitsRgone. You are putting cream X on one side of your face and cream Y on the other and you don't know what's in either cream. Furthermore, we will not let you evaluate the results. You may have come to believe that cream X is the ZitsRgone and that belief may affect your evaluation. So, we will have several independent observers evaluate your cheeks and rate them before and after for "pimplatude" (quality and quantity of pimples). The independent evaluators will not be told which cream is the ZitsRgone, so their judgment will not be influenced by their beliefs or knowledge. After they have rendered their verdicts, we reveal whether cream X or cream Y is the ZitsRgone.
If ZitsRgone is a signifant causal factor in the reduction of pimplatude, then our results should show that there is a significant difference in pimplatude between the two cheeks, such that the ZitsRgone treated cheek has a significantly lower pimplatude rating than the the control cheek. If so, can we now declare that ZitsRgone works? No, unfortunately our study sample is too small to draw any significant conclusions from. We need to do a large scale study to make sure than the difference we find is not due to chance.
Now, we should recognize that we are going to have a problem testing a hypothesis such as "nothing happens according to chance." A control study assumes that some things happen according to chance and some don't. If it were true than nothing happens by coincidence then we could not test any causal claims.
But now we have a problem. Our list of potential claims to be tested includes one which not only can't be tested itself but which implies that none of the others can, either. However, it seems possible to test at least one of claims, namely, the claim that all events are caused by will power. If it is true that we can test this claim, then it would logically follow that the theory of manifesting consists of at least one false claim and is inherently contraditory. Generally, reasonable people do not consider it wise to believe in false or contradictory claims.
But is it really possible to test the claim that all events are caused by will power? Well, if it is true that all events are caused by will power, then it must be true that some events are caused by will power. So, we can restrict ourselves to a particular type of event and test a more specific hypothesis which, if true, would confirm though not prove, our more general hypothesis. For example, we might try to test the hypothesis that through an act of will a person can cause another person to die. What we might do is develop a list of 2,000 famous people between 60 and 70 years old, and randomly divide the list into lists of 1,000 each (List A and List B). We randomly select 2,000 people to be in our experiment and randomly divide them into two groups. Each person in Group A is given a name randomly selected from List A and is trained to use manifesting techniques to will the death of the person whose name he or she has been given. Each person in Group B is randomly assigned a name from list B, but they are not told this. Those in Group B are lied to and told that they are on a waiting list for some great study in human potential.
Let's give our manifesters ample time to accomplish their wilful acts of murder by thought. After a year is up, we check List A and see who on the list has died during the past year. Let's say, for the sake of argument, that 100 people on List A have died during our experiment. Does this mean that manifesting is only 90% effective? Of course not. If manifesting really works, no one on the list should still be living. Still, 10% have died. Doesn't that prove that at least some people can manifest? Not really. Remember that we have matched the people on List B to members of Group B. If about 10% of those on List B died, then that would indicate that the deaths of those on List A were not caused by manifesting, but were coincidental with the manifesting.
Now, how would you test the hypothesis that people create their own luck? Or the claim that all events are part of a plan? The latter is surely a metaphysical claim and incapable of empirical testing. The former seems to depend upon the metaphysical belief that nothing happens by coincidence, so it too would be intestable.
The fact that a claim cannot be empirically tested does not mean it is false, however. But it does mean that it is not empirical. And, in the case of belief in manifesting, it appears to be manifestly the case that to put your faith in such a belief one must maintain contradictory notions and/or give up the belief that any causal hypothesis can be tested.
further reading
Giere, Ronald, Understanding Scientific Reasoning, 2nd ed,(New York, Holt Rinehart, Winston: 1984.
Kourany, Janet A., Scientific Knowledge: Basic Issues in the Philosophy of Science, Belmont: Wadsworth Publishing Co., 1987.
Sagan, Carl. The Demon-Haunted World: Science as a Candle in the Dark, New York:Random House, 1995.